Simple Headend System MODBUS

Introduction

Energy data collection from meters is a critical operation for energy companies, requiring reliable, scalable, and automated solutions. The Headend System addresses these needs by providing a robust infrastructure for meter data collection and processing.

Business Value

The Headend System provides organisations with automated, reliable energy data collection from their meter infrastructure. This solution delivers:

Automated data collection from smart meters
Scalable architecture for managing multiple meters
Real-time data processing and validation
Reduced manual intervention in meter reading
Comprehensive audit trail for compliance and troubleshooting

System Overview

The Headend System streamlines the process of collecting and managing energy usage data (load profiles) from SEL smart meters. It employs industry-standard MODBUS protocols for meter communication and structures the retrieved data for analysis. The system is built on two primary components that work in tandem to ensure efficient data collection:

The Meter Scheduler: Manages meter discovery and scheduling
The Meter Processor: Handles parallel data collection and processing

Meter Scheduler:

1. Identifies Active Meters: The Scheduler uses the EWX API to find meters(datasource) with the tag "headend" and marked as “active.” (see section: SettingUp Your Meters)

2. CreatesTriggerMessages: For each active meter, the Scheduler generates a trigger message. Bases on the tag information retrieved from Energyworx. This message is created with the following structure:

class Trigger(msgspec.Struct): """Output object to send to the Queue.""" meter_id: str terminal_id: str meter_serial: str meter_address: str meter_port: str ldp_recorder_nr: str recorder_freq: str

3. Queues Messages: Trigger messages are placed in a queue ('sel-read-trigger') to be processed by the Meter Processor.

Meter Processor:

1. RetrievesTriggerMessages: The Processor collects batches of trigger messages from the queue.

2. Reads Meter Data in Parallel: Using the information inthe trigger messages, the Processor connects to each meter in the batch simultaneously. It then executes the following steps for each meter:

Retrieves Meter Information: This includes the serial number, uniqueidentifier, and terminal ID. This data can be used to update or validate meter information in the system.
Reads LoadProfile Data:
Iterates through the specified load profiles.
Fetches the name of each load profile.
Determines the reading frequency of the profile from the meter or cache.
Calculates the starting point for data reading based on the last recorded time.
Repeatedly reads rows of data until the required number of data points is reached, considering the elapsed time since the last read.
Organizes the read data into a structured format.

3. Combines and Sends Data: The Processor consolidates all collected data,sorting it by its corresponding ewx namespace_id_, and sends it to theFile Manager of the relevant namespace for storage, ingestion and further analysis.

Output Data

The resulting output is a well-structured JSON file, containing comprehensive information about each meter and its load profiles:

    [
      {
        "METER_ID": "735-WA-F8",
        "METER_SERIAL": "3221380863",
        "TERMINAL_ID": "3221380863",
        "READS_TS": "2024-07-12T03:00:00",
        "LOAD_PROFILES": [
            // ... (Detailed load profile data)
        ]
      },
      {
        "METER_ID": "735-WA-F3",
        // ... (More meter data)
      }
    ]

Auditing and Logging

All processing activities are recorded in the sel_processing_logs BigQuery table, providing a detailed audit trail for troubleshooting and analysis.

Setting Up Your Meters

To use and or test the headend system,you need to setup meters(datasources) in Energyworx. To do so, follow the below steps.

1. Gather Meter Information:

Collect the following details foreach meter you want to integrate into the system:
Meter Name: A unique name for easy identification (e.g.,"735-LW1").
Timezone: The geographic location of the meter (e.g.,"US/Pacific").
Tag****Name: A consistent tag to identify these meters within the system (e.g.,"headend").
Meter Details: The meter's ID, serial number, and terminal ID.
ConnectionInformation: The meter'sIP address and port number (crucial for establishing communication).
Load Profile(LDP) Recorder Numbers: Specify the specific load profiles (data sets) you want to collect (e.g.,"1,2").
RecordingFrequency: How often the meter records data in minutes (e.g., "5").
Active Status: Indicate whether the meter should be read immediately("True") or later ("False"). This allows you to prepare the system for future meters.

2. Create Configuration File:

Organize the collected meter information into a structured JSON file. Here's an example:

    [
      {
        "meter":"735-LW1",
        "timezone": "US/Pacific",
        "tags": {
          "tag_name": "headend",
          "valid_from": "1899-31-12 4:00:00",
          "meter_id": "735-LW1",
          "meter_serial": "000",
          "terminal_id": "1.0.0",
          "meter_address": "216.110.193.236",
          "meter_port": "10084",
          "ldp_recorder_nr": "1,2",
          "recorder_freq": "5",
          "active": "False"
        }
      }
    ]

3. Ingest Configuration File:

Import the configuration file into the system.
The system will automatically create data sources named after each meter and corresponding tags based on the tag_name field.
Remember, the meter_address , meter_port , ldp_recorder_nr ,and active fields are the most critical for proper setup.

Known limitations:

1. Decimal precision:

Data is always being retrieved with 2 decimal precision. The 2 decimal precision is a limitation of the SEL Modbus meter protocol. When a meter is configured to record meter data on kWh level but needs more precision, the meter can for instance be configured to record data with Wh precision. In Energyworx this can be converted to kWh with 5 decimal precision.

Introduction​

Business Value​

System Overview​